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ABSTRACT 


Responding to globalization trends, many engineering schools are internationalizing their courses 
and curricula to prepare graduates for careers that involve working across countries and cultures. As 
a result, both students and staff are looking beyond study abroad to international work, research, and 
service learning opportunities as alternate pathways for high-impact experiential learning in global 
context. However, such programs can also pose unique challenges related to preparing students for 
professional practice abroad and systematically evaluating program outcomes and impacts among 
students and other stakeholders. This paper responds to these challenges by reporting on International 
Research and Education in Engineering (IREE) 2010 China, a summer engineering research abroad 
program. It describes orientation strategies designed to improve student readiness for global practice, 
and presents mixed-method assessment methods and results for global engineering competency and 
other key learning outcomes. The main audience for this paper includes faculty and staff who develop 
and support global learning experiences in engineering and other professional degree programs. 
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INTRODUCTION 


Over the last decade, international education has gained significant prominence and visibility 
in many professional schools, in part reflecting broader globalization dynamics and increasing 


WINTER 2014 


1 




ADVANCES IN ENGINEERING EDUCATION 

Developing Globally Competent Engineering Researchers 


demand for a globally competent workforce. Bousquet (2003), for example, summarizes interna¬ 
tionalization trends and drivers in schools of business, engineering, and medicine, while Ferguson 
(2010) documents similar movements in law schools. Nonetheless, there remain many challenges 
associated with expanding participation in global education programs among students pursuing 
professional degrees. In fact, NAFSA: Association of International Educators has identified “science 
and technology majors” and “students from professional schools” as among populations of special 
interest in ongoing efforts to scale up education abroad (NAFSA, n.d.; Anderson, 2009). Specific 
difficulties faced by these groups include fitting study abroad and related coursework into already 
full schedules, transferring academic credit from institutions abroad, a lack of support from faculty 
and administrative staff, and weak connections between professional schools in the U.S. with their 
counterpart institutions abroad (Bousquet, 2003). 

Similar patterns are evident in the field of engineering education. For more than a decade, many 
influential stakeholders have argued that global competency is crucial for cultivating a new gen¬ 
eration of “global engineers” who are ready to work an increasingly diverse, interconnected, and 
rapidly changing world (e.g., Boeing, 1997; Wulf, 2003; Katehi, 2005; Sigma Xi, 2007; Grandin & 
Hirleman, 2009). Yet as many of these same reports explain, most degree courses and programs 
fail to produce truly global engineers, leading the authors to call for reforms. Parkinson (2007) 
has estimated that only about 7.5% of engineering students study abroad, while Shuman adds that 
just 10-15% of engineering schools are taking global education seriously (Bremer, 2008). Another 
group of reformers noted the lack of a widespread tradition of “sending engineers to study or work 
abroad,” and they went on to identify sixteen contemporary obstacles to scaling up global engineer¬ 
ing education (Grandin & Hirleman, 2009). 

Historically, study abroad programs have been the primary means by which students in engineering 
and most other fields add an international dimension to their education. In addition, many organiza¬ 
tions and institutions have tended to report international education trends by tracking only those 
students who receive academic credit for their participation in global programs, which further exag¬ 
gerates the visibility and prominence of study abroad. To be sure, study abroad and other for-credit 
activities are essential parts of the global learning portfolio. However, another important strategy 
for promoting global learning experiences among students in engineering and other professional 
schools centers on looking beyond traditional credit bearing study abroad programs and toward 
curricular or extracurricular work, research, and/or service learning experiences abroad. 

These alternative program formats offer participants a wider range of opportunities for relevant, 
experiential learning, including hands-on and/or project-based work situated in global context. Pro¬ 
grams of this type also have other features that can appeal to many professional school students, 
such as better scheduling flexibility, work positions that may include some sort of salary, research 
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options for graduate students, and less problems with determining course-to-course equivalencies 
for credit transfer. As a result, many professional schools are expanding their global offerings to 
include programs of this type. Yet in doing so they face many new challenges, such as getting stu¬ 
dents ready for these unique experiences, identifying sources of funding, sustaining cross-national 
relationships, and systematically evaluating learning outcomes and other kinds of impacts. 

As a contribution to the literature on global professional learning, this paper reports on Interna¬ 
tional Research and Education in Engineering (IREE) 2010 China, an engineering research abroad 
program. More specifically, it discusses orientation strategies designed to better prepare engineers 
for professional work abroad, as well as leading-edge assessment techniques to measure how 
programs of this type impact student learning and develop professional competence. Our work is 
unique given its discussion of a program serving both undergraduate and graduate students, along 
with its specific focus on global engineering competency. As discussed in more detail below, we 
propose that this construct is comprised of a variety of attributes that are particularly relevant for 
engineering practice in cross-national/cultural contexts. 

The literature review that follows is focused on program formats and assessment strategies for 
international education. We begin with a discussion of general trends in professional schools to 
help show how our efforts are situated and relevant in a wider context. We then turn to engineering 
more specifically, including strategies for developing and assessing global engineering competency 
and related capabilities. This is followed by a summary description of the IREE 2010 China program, 
with emphasis on orientation content and learning activities. The final sections of the paper discuss 
assessment methods and results related to participants’ perceived readiness to go abroad, foreign 
language learning, cross-cultural competence, and global engineering competency. The paper con¬ 
cludes with a brief discussion of best practices for faculty and staff who develop and support global 
learning experiences in engineering and other professional degree programs. 


LITERATURE REVIEW 


International Education in the Professions: Learning Outcomes, Program Formats, 
and Assessment Strategies 

Data collected by organizations like Institute for the International Education of Students (IES 
Abroad) and Institute of International Education (HE) provide important evidence regarding ben¬ 
efits often associated with participating in educational experiences abroad (Dwyer & Peters, 2004; 
J. Walter Thompson Education, 2003). Many of these benefits are in turn frequently linked to the 
concept of “global competency.” As summarized by participants at the 2007 Lincoln Commission 
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Forum on Study Abroad and Economic Competitiveness, “Studies show that business has a growing 
expectation of new employees having global competency, and that there is a growing appreciation 
by employers of the strong correlation between study abroad and global competency” (Stearns, 
2009, p. 68). The extant literature on international education also reveals many other attributes 
frequently associated with educational experiences abroad, such as intercu/tural competency, inter- 
cultural communication competence, worid-mindedness, cross-cultural knowledge/awareness, open- 
mindedness, and global citizenship, to name a few (Byram & Nichols, 2001; Beamer 2011; Douglas & 
Jones-Rikkers, 2001; Deardorff, 2009). The specific concepts used to define global competency and 
related constructs vary considerably depending on the program and discipline, and the associated 
terminology is often used ambiguously and interchangeably. 

Definitional issues are among a longer list of challenges faced when studying the impacts and 
outcomes of global experiences. In fact, many discussions of international education use the word 
assessment rather loosely and broadly. In this paper we distinguish between program evaluation 
and assessment of learning outcomes. While both are important to the success of a study abroad 
program, each approach involves partially distinct data and goals. Program evaluation typically looks 
at overall measures of program quality and success, including various participation benchmarks 
(e.g., how many students in what programs and to what destinations) and perceptions of program 
quality (e.g., program format, logistics, student support, financial aspects, etc.). Evaluation studies 
are often focused on gauging the overall success of a program, while potentially fulfilling various 
institutional reporting requirements. 

In contrast, we understand assessment in education abroad as focused on measuring specific 
learning outcomes among participants, including those related to global competency. Approaches 
to assessing such outcomes vary widely and often fall short (e.g., Deardorff, 2009a; 2009b). Many 
of the instruments employed by faculty and administrators are custom developed for specific pro¬ 
grams and thus lack content validity and empirical strength. Another prominent pitfall in assessing 
learning outcomes involves overreliance on students self-rating their proficiency or competence in 
specific areas, where it is often unclear whether there is a shared understanding of how outcomes 
are defined, much less whether or not respondents possess the abilities they report (Podsakoff 
& Organ, 1986). In other instances, program staff and researchers struggle to find and implement 
methods that can detect meaningful changes in global competency resulting from various kinds of 
global learning experiences (Stearns, 2009, Ch. 9). 

These difficulties are not surprising, especially given the complexity of the constructs involved 
and the large number of extant papers and tools focused on global competency. A comprehensive 
overview of theories and research on intercultural competence by Spitzberg and Changnon (2009), 
for example, identified more than a hundred instruments available to measure many different facets 
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of intercultural competence and related constructs. Along similar lines, Fantini and Tirimizi (2006) 
identified 69 instruments that measure various aspects of intercultural competence, e.g., tolerance, 
foreign language proficiency, individual-collective orientation, openness, and so on. Given such 
challenges, programs often adopt assessment tools that are readily available or easy to deploy, 
even if they are not well aligned with their participant populations and/or the intended program 
learning outcomes. 

Nonetheless, over the last decade a handful of quality instruments have gained considerable 
prominence in ongoing efforts to measure intercultural competence and related constructs, including 
through the efforts of a number of international programs run by professional schools. Representa¬ 
tive examples include: Intercultural Development Inventory (IDI), which assesses the extent of an 
individual's intercultural development along a continuum ranging from extreme ethnocentrism to 
ethnorelativism (Hammer, Bennett, & Wisemen, 2003); Beliefs, Events and Values Inventory (BEVI), 
which measures whether, how, and to what degree people are (or are likely to be) "open” to various 
transformational experiences such as studying abroad (Shealy 2004, 2005); and the Cross-cultural 
Adaptability Inventory (CCAI), which evaluates some key qualities associated with cross-cultural ef¬ 
fectiveness, including personal autonomy, perceptual acuity, flexibility and openness, and emotional 
resilience (Kelley & Meyers, 1992). As this summary suggests, each of these instruments assesses 
different dimensions of intercultural competence. This can in part be attributed to the complexity 
of the constructs involved, as well as a general lack of consensus regarding how intercultural com¬ 
petence and related concepts might best be defined. 

The professional schools face further challenges related to defining and assessing key learning 
outcomes that are partially or wholly specific to their fields, especially given the wide variety of global 
program formats open to students. Since at least the 1980s, for example, international education 
has gained considerable momentum in the business and management fields, and coverage of inter¬ 
national business and globalization topics is increasingly the norm in most degree programs (Kwok, 
et al., 1994; Bousquet, 2003; AACSB, 2011). In fact, by 2009-2010 business/management students 
represented 20.8% of all American students studying abroad, second only to students in the social 
sciences (22.3%) (HE, 2011), while a 2009 study found that 55 of 70 (or almost 80%) of surveyed 
MBA programs had some kind of global requirement (DAngelo, 2012). Interest also continues to 
grow around providing business and management students with experiential learning opportunities 
through work and/or study abroad (Edwards et al., 2003; AACSB, 2011), and evidence suggests that 
short travel programs and virtual teamwork experiences are prevalent formats, especially at the 
Master’s degree level (Forray & Woodilla, 2011). 

Yet despite the apparent relevance of global themes in business and management schools, 
there remains a lack of valid and reliable strategies to specify and assess related facets of global 
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competence. One literature review of leading business journals from 1976-2004 revealed a lack of 
agreement about how experts in the field define cross-cultural competence (Johnson, Lenartowicz, 
& Apud, 2006). They also noted a lack of in-depth studies focused on specific knowledge, skills, and 
attitudes underlying cross-cultural competence. A recent report from the Association to Advance 
Collegiate Schools of Business (AACSB) similarly points to both definitional and assessment chal¬ 
lenges as business schools internationalize, including too little attention paid to identifying specific 
learning outcomes related to international competencies, conceptual ambiguity around popular 
terms like “global mindset,” and a tendency to narrowly focus assessment on knowledge gains rather 
than changes in values and attitudes (AACSB, 2011). 

As many recent reports indicate, the impacts of globalization are also evident in many U.S. medi¬ 
cal schools (Chase & Evert, 2011). For example, the number of medical students reporting partici¬ 
pation in “global health experiences” doubled from 15% in 1998 to more than 30% in 2011 (Chase & 
Evert, 2011; AAMC, 2011). The field also has its own particular array of program formats, including 
somewhat less emphasis on formal coursework abroad in favor of opportunities for global learning 
through student and other non-profit organizations, global health initiatives, and clinical rotations 
at institutions abroad (Drain et al., 2007; Chase & Evert, 2011). Other typical activities include train¬ 
ing in foreign language and culture, working in health clinics and community outreach programs, 
performing comparative studies of U.S. and foreign medical systems, and deploying and maintain¬ 
ing medical devices. Opportunities for research abroad are also reasonably common in graduate 
medical education (Chase & Evert, 2011; McKinley et al., 2008). 

While such programs provide medical students with meaningful and eye-opening experi¬ 
ences, efforts to systematically measure related learning outcomes remain underdeveloped. For 
instance, Mustchnick, Moyer, and Stern (2003) conducted a review of forty-two papers on the 
impacts of cross-cultural exchanges on medical and nursing students. While most of the papers 
they surveyed described increases in global competence and personal development among par¬ 
ticipating students, they largely failed to demonstrate clear benefits to those who participated 
in international education as compared to those who did not. In fact, the paper cites only three 
studies that comment on the validity of the assessment tools used, with much of the rest of the 
literature relying on subjective, self-reported outcomes. The authors conclude by calling for fur¬ 
ther investigation of behavior-based outcomes, including those that utilize patient evaluation 
and patient satisfaction scales to more systematically and rigorously measure the effectiveness 
of professionals-in-training. As another notable contribution in this space, a recent guidebook 
on Global Health Training in Graduate Medical Education proposes six specific areas of compe¬ 
tence for students in residency global health training programs (Anspacher et al., 2011). While 
the authors tend to emphasize self-assessment strategies in their report, they do list a number of 
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more rigorous approaches such as knowledge tests, direct observations of performance, patient 
surveys, 360-degree evaluations, etc. 

As international education continues to expand and diversify to suit the needs of the professional 
fields, there will likely be increasing need for definitional clarity of learning outcomes and associated 
assessment tools that are valid, reliable, and scalable. In addition to measuring global competency 
in general, effective assessment strategies must also investigate how different kinds of program 
formats and learning experiences are contributing to the formation of global professionals. 

Global Engineering Education: Learning Outcomes, Program Formats, and Assessment Strategies 

Historically, the impact of internationalization trends on most engineering degree programs has 
been modest. However, there is evidence of a long tradition of engineering students engaging in 
many kinds of experiential learning abroad, from volunteer activities and service learning projects 
to working in foreign labs, firms, or branch offices (Jesiek & Beddoes, 2010). Many contemporary 
discussions of global engineering education have acknowledged and discussed these different 
modes of global learning. For example, Downey et al. (2006) present a typology of methods for 
achieving global competency, including international work placements. Along similar lines, Par¬ 
kinson’s (2007) review of global learning opportunities in engineering mentions intern/co-op and 
research abroad, including specific examples of programs in each category. Also of note is a recent 
paper by Arzberger et al. (2010) that compares and contrasts four programs that provide science 
and engineering students with research and/or study abroad opportunities. 

While a comprehensive review is beyond the scope of this paper, it is worth discussing some of 
these initiatives in more detail to gain further insights regarding the kinds of orientation and assess¬ 
ment strategies utilized by such programs. For instance, Arzberger et al. (2010) present an in-depth 
profile of Pacific Rim Experiences for Undergraduates (PRIME), a summer research abroad program 
that from 2004 to 2009 sent more than a hundred students for nine-week research placements in 
the Pacific Rim. Pre-departure training included a two-hour session on culture, and students were 
asked to write weekly responses to reflection questions while abroad. In discussing assessment 
strategies and metrics, the organizers emphasize cultural awareness as a key outcome, and present 
preliminary results suggesting positive pre/post-travel trends in Intercultural Development Inventory 
(IDI) scores for 21 PRIME participants. 

A somewhat more ambitious approach is evident in NanoJapan, a summer nanotechnology re¬ 
search abroad program for undergraduate students based at Rice University. To begin, participants 
are required to complete a robust three-week orientation program in Japan that covers: relevant 
technical topics; language, including technical Japanese; and an introduction to Japanese culture, 
society, and history (Matherly, Phillips, & Kono, 2011). This is followed by eight-week placements in 
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university research labs across Japan. The program’s ambitious assessment plan includes data col¬ 
lected using the Pittsburgh Freshman Engineering Student Attitude Assessment (PFESAA), Oral 
Proficiency Interview (OPI) for language capabilities, weekly blog posts, and longitudinal tracking 
of graduates (NanoJapan, n.d.). However, little in the way of results has yet been reported. 

As part of its International Plan (IP), Georgia Institute of Technology has also facilitated expanded 
global learning opportunities for its engineering students, including research and work abroad 
placements that can be used by students to fulfill some program requirements. Explicitly framing 
their approach to global education as oriented toward “competency” rather than “activities,” IP 
has been intentionally designed as a structured and multi-faceted intervention that lives within the 
context of existing degree programs (Gordon & Lohmann, 2008). The evaluation framework for the 
program is also ambitious and robust, involving multiple measures of second language proficiency, 
comparative global knowledge, intercultural assimilation, and intercultural sensitivity (Lohmann, 
Rollins, & Hoey, 2006). While findings have not yet been widely published, early results presented 
in institutional reports and at conferences suggest substantial improvements in IDI scores among 
participants, as well as evidence of enhanced career satisfaction and success (Georgia Institute of 
Technology, 2011; Gordon & Henry, 2011). 

The University of Rhode Island’s International Engineering Program (IEP) also has an internship 
abroad requirement for participating students as they work toward dual Bachelor’s degrees in 
engineering and foreign language (German, French, Spanish, or Chinese) (Grandin, 2006). Faculty 
and administrators who have been involved with the program have generated a number of notable 
publications reporting student outcomes, including a book by Grandin (2011) that profiles fifteen 
graduates of the program, largely based on interviews. In addition, Erickson (2011) reports on exit 
survey and interview data collected from eleven IEP graduates, placing particular emphasis on lan¬ 
guage competency, technical abilities, and cross-cultural sensitivity. 

While further review of programs offering students research or work placements abroad is beyond 
the scope of this paper, other examples worth noting include the University of Cincinnati’s well-known 
International Co-op program (Elliot, 2006) and Purdue University’s Global Engineering Alliance for 
Research and Education (GEARE) (Allert et at, 2007; Groll & Hirleman, 2007; Chang, Groll, & Hirle- 
man, 2011). Many individual students and universities have also benefited from services provided 
by the International Association for the Exchange of Students for Technical Experience (IAESTE), 
which has been helping American students find technical internships abroad since at least the 1950s 
(Jesiek & Beddoes, 2010). NSF has played an important role in this area by providing support for 
undergraduate and graduate research abroad under the auspices of International Research Experi¬ 
ences for Undergraduates (IREUs), East Asia and Pacific Summer Institutes (EAPSI) for U.S. Graduate 
Students, Partnerships for International Research and Education (PIRE), and other programs. 
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As this overview suggests, there is considerable diversity in the preparation and orientation strate¬ 
gies employed by programs that send engineering students abroad for research and/or work, ranging 
from relatively brief one-time training sessions to immersive, multi-week (or even multi-semester) 
courses of study. Yet there is little evidence regarding the variable effectiveness of preparatory ac¬ 
tivities depending on their scope, duration, and content. Investigating such variables is made even 
more challenging given that it is often very difficult to separate the impacts of orientation from 
other kinds of program components, activities, and experiences. 

Additionally, the extant literature on outcomes and assessment reflects a historical emphasis 
on conventional technical skills, along with various capabilities typically associated with globally 
competent individuals from most any professional field, such as language skills, and intercultural 
competency and related constructs. Much less attention has been paid to evaluating what we call 
“global engineering competency,” which includes a range of attributes uniquely or particularly 
relevant for practicing engineering in cross-national/cultural contexts. Johri and Jesiek (in press) 
propose a preliminary list of thematic areas that may in part constitute this domain, including: global 
engineering ethics, cross-cultural technical teamwork, international standards and regulations, and 
knowledge brokering. These authors also draw on the work of Downey et al. (2006) to discuss related 
attributes in the area of “engineering cultures,” including the ability to understand and work in the 
midst of patterned differences in engineering work practices across cultures, countries, and regions. 
Parkinson (2009) similarly points to the importance of being able to “[u]nderstand implications of 
cultural differences on how engineering tasks might be approached” (p. 11). 

Assessment strategies targeting these kinds of specialized yet very important areas of compe¬ 
tence remain underdeveloped. As Lohmann et al. note in their discussion of global competency in 
engineering: “Largely absent are studies featuring rigorous methods for assessing ... competencies 
specifically related to professional practice within the academic discipline” (2006, p. 125). In re¬ 
sponse, Georgia Tech has been assessing “global disciplinary practice” through employer surveys 
and rubric-based evaluations of senior design projects. However, little has yet been reported on the 
success of these approaches, and their portability and scalability remain unclear. 

Other related work includes Ragusa’s Engineering Global Preparedness Index (EGPI), a 30-question 
survey covering four subscales: engineering ethics, engineering efficacy, engineering global-centrism, 
and engineering community connectedness (Ragusa, 2011). Although early results show promise, 
this instrument’s basis in global citizenry theory may restrict its relevance to certain contexts and 
program types (e.g., academic rather than industry settings), and its validity is limited by its reliance 
on self-efficacy data. Downey et al. (2006) have taken a different approach to assessing students 
in an undergraduate elective course (n = 146) that aims to develop global competency. Their open- 
ended scenario-based instrument is designed to evaluate the ability of students to “explain how 
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national differences among engineers are important in engineering work” (Downey et al., 2006, 
p. 117). A pre/post-course multiple-choice quiz was also used to measure attainment of course 
content knowledge, and a final course survey asked students to self-report their gains related to 
specific course learning outcomes. This research showed significant pre/post-course increases in 
student attainment of key aspects of global engineering competency. Amelink et al. (2012) have 
probed some similar themes in a short-term engineering study abroad program through analysis of 
qualitative data collected via a pre/post-course writing assignment and a focus group session. 

In part building on the work of Downey et al. (2006), this paper represents another contribution 
to the small but growing body of literature discussing training and assessment strategies for global 
engineering programs that send students abroad for research and/or professional work. In the sec¬ 
tions that follow we turn to a more detailed description of the IREE 2010 China program, followed 
by a discussion of our assessment methods and findings. 


IREE 2010 CHINA: PROGRAM OVERVIEW AND INSTRUCTIONAL STRATEGIES 


Initiated in 2006 by NSF Divisions in the Directorate for Engineering (ENG) and Office of Interna¬ 
tional Science and Engineering (OISE), the International Research and Education and Engineering 
(IREE) program originally had two main objectives: 1) “providing early-career researchers in engi¬ 
neering with international experience in research and education,” and 2) "enhance and broaden engi¬ 
neering research and education activities in current engineering awards by initiating closer linkages 
between awardees and their foreign counterparts” (NSF, 2006). During its first two cycles (FY2006 
and FY2007), IREE was run as a supplemental funding program, where existing NSF grantees could 
apply for additional financial support to send early career engineering researchers (including stu¬ 
dents, post-docs, and faculty) to collaborating institutions outside the U.S. for medium-duration 
visits (3-6 months). A total of 247 awards were made during this period, supporting grantee travel 
to collaborating institutions in 45 different countries (Chang & Hirleman, 2008; Chang, Atkinson, & 
Hirleman, 2009). 

The focus of IREE shifted in 2010 when a team from Purdue University was awarded a grant (NSF 
EEC-0965733, “IREE: Developing Globally Competent Engineering Researchers”) to administer a 
new version of the program. The original IREE objectives - namely, promoting international research 
collaborations and developing global competency among participants - were coupled with a man¬ 
date to set the stage for a future scale-up, including by systematically evaluating and studying all 
aspects of the program. Other major changes for IREE 2010 included limiting research placements 
to China and restricting eligibility to undergraduate and graduate students. Additionally, participants 
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were given two options for research placements. In line with the original IREE framework, a “self¬ 
placement” option allowed applicants to propose going to a specific host institution or lab, often 
identified with the help of a faculty advisor or mentor. In addition, a new “site-placement” option 
was added, allowing IREE staff to arrange placements for some awardees at select partner sites 
and labs. 

Intensive promotion of the IREE 2010 China program led to receipt of more than 360 applications. 
A group of program staff reviewed 278 complete applications (i.e., those not missing any materials) 
to develop a ranked list of prospective awardees based on ten evaluation criteria (e.g., quality of 
personal statement, academic qualifications, quality of recommendation letters, evidence of en¬ 
thusiasm for working in China, etc.). A total of 58 participants ultimately accepted an IREE grant. 
After completing orientation activities in May 2010, participants traveled to China for 10-12 weeks to 
work on frontier science and engineering research projects in university, industry, and government 
labs. Grantees were also required to participate in a re-entry meeting held about one month after 
they returned to the U.S. (September 25-26, 2010 in Chicago). The total award amount averaged 
approximately $7,500 (undergraduate students) or $8,500 (graduate students) for all research 
stipends and reimbursable expenses, and another $1,400 in additional expenses that were covered 
directly by the program. 

Given the focus of this paper, it is worth describing in more detail how the program supported 
development of global competency among participants. One centrally important feature of IREE 
was an extensive orientation curriculum that the IREE team developed and ran in three differ¬ 
ent formats during May 2011. Of the 58 grantees: (i) 19 students were hosted by the IREE team 
for a two-week orientation session in Shanghai, China; (ii) 19 students were hosted by the IREE 
team for a two-week orientation session at Purdue’s main campus in West Lafayette, Indiana; and 
(iii) 20 students were assigned to a flexible five-week online orientation program, allowing them to 
participate from their own workplace or residence. All orientation programs offered instruction in 
Chinese language (Mandarin), including nine half-days of instruction for face-to-face orientation 
groups, and roughly equivalent coverage recommended for the online group using an introductory 
Chinese language textbook and accompanying audio lessons (Jianfei, 2004). In addition, Chinese 
culture and history were introduced to students in five half-days of instruction for the face-to-face 
groups, and reading assignments paired with online discussion activities for the online group. Both 
formats utilized an appropriate and accessible introductory textbook (Morton & Lewis, 2005). 

Finally, participants were taken through an Engineering Cultures China curriculum, which was 
delivered to the Purdue and Shanghai groups during four half-day sessions, and to the online 
group via a comparable series of multimedia lectures and accompanying discussion forum activi¬ 
ties. These materials were in part based on an instructional model originally developed by Downey 
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• Engineering Cultures China - Introduction* 

• Science and Technology in Ancient China 

• Engineering Education and Profession in Early Modern China 

• History of Engineering Education, 1911-1976* 

• Engineering and Engineering Education in the Post-Mao Era* 

• Interview Excerpts - Perspectives on Chinese Engineering, Education, and Culture 

• Imitation or Innovation* 

• Engineering Problem Solving with People* 

^Available as narrated PowerPoint presentations on GlobalHUB.org (Jesiek, 2010) 


Table 7. Engineering Cultures China Learning Modules. 


and Lucena (Downey et al., 2006; Downey, 2008) and used to support production of online learn¬ 
ing modules focused on a number of different country contexts (Downey & Lucena, 2008). The 
new Engineering Cultures China curriculum that we developed for the IREE program provided 
grantees with a wealth of information about the historical development and contemporary state 
of engineering education and the engineering profession in China, as well as specific participant 
observation methods and strategies to enhance their ability to work more effectively in diverse 
global contexts. Table 1 gives titles for each discrete Engineering Cultures China presentation, and 
also indicates which modules are freely available online. Some existing secondary sources were 
used to develop and scaffold the module, including Andreas’ (2009) book titled Rise of the Red 
Engineers: The Cultural Revolution and the Origins of China’s New Class. The creation of instruc¬ 
tional materials is also leading to a series of derivative publications on the history of engineering 
in China (e.g., Jesiek & Shen, 2012). 

A number of strategies were used to promote reflective learning and support learning outcomes 
among students throughout their IREE experience. For example, students in the face-to-face orien¬ 
tation groups were asked to write and perform a brief dialog or script portraying a small group of 
American and Chinese engineers working collaboratively on a project or problem, with the goal of 
helping them think about how engineers from diverse backgrounds may define and solve problems 
differently. Additionally, during their research placements participants were twice asked to respond 
to a set of reflective writing prompts in blog/journal entries posted to the GlobaIHUB web site. As 
summarized in Table 2, these prompts were based on a critical incident framework that encourages 
individuals to proactively identify, analyze, and respond to specific challenges they face (Crisp et al., 
2005; Hanson & Brophy, 2012; Walther et al., 2011). Still other opportunities to productively reflect 
on the IREE experience were enabled through a variety of activities at the required re-entry meeting 
(e.g., interviews, focus groups, etc.), as well as a final trip report that encouraged participants to 
write about the broader impacts of their experience. Select blog posts and trip reports from IREE 
2010 grantees are available on GlobaIHUB ( http://qlobalhub.org/iree ). 
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1. Since starting your research assignment, what is ONE significant challenge you have SUCCESSFULLY faced or 
resolved? In your 1+ paragraph response, make sure you: 

a. Describe the challenge. 

b. Discuss what you have learned from this experience. 

c. Reflect on how this experience might help you in the future. 

2. Since starting your research assignment, what is ONE challenge you have NOT been able to resolve? In your 1+ 
paragraph response, make sure you: 

a. Describe the challenge. 

b. Discuss what you have learned, or not learned, from this challenge. 

c. Reflect on how you will deal with this type of challenge in the future. 

3. During the past 2-3 weeks, what is one trait/practice/behavior/way of life that you have observed in China that is most 
different from your native culture/background? In your 1+ paragraph response, make sure you: 

a. Describe the trait/practice/behavior/way of life. 

b. Discuss similarities/differences compared to your own culture/background. 

c. Reflect on how you reacted to this trait/practice/behavior/way of life. 

4. Identify and describe one specific goal for personal and/or professional development that you hope to achieve within 
the next month. 


Table 2. Reflective Writing Prompts for iREE Participant Blogs/Journals. 


ASSESSMENT METHODS 


Participant Demographics 

Among IREE awardees, 27 (or 46%) were women, 5 (or 9%) self-identified as underrepresented 
minorities (Hispanic/Latino or African American), and 14 (or 24%) as Asian. A total of 55 grantees 
(or 95%) were U.S. citizens, 2 (or 3.5%) were U.S. permanent residents, and one (or 1.7%) was a Ni¬ 
gerian citizen with a U.S. visa. At the time of application, 28 awardees (or 48%) were undergraduate 
students and 30 (or 52%) were graduate students. Awardees represented more than 40 different 
home universities in the U.S., and their most common home department affiliations were: electrical/ 
computer engineering (13 participants), biomedical engineering (7), civil engineering (7), mechani¬ 
cal engineering (6), and environmental engineering (5). There were 37 self-placement and 21 site- 
placement participations, with the largest number of students at Shanghai Jiao Tong University, 
Tsinghua University, and Xi’an Jiao Tong University. 

Data Collection and Analysis 

As summarized in Table 3, we used a variety of strategies to collect a wealth of research and 
assessment data during every major program phase, with highlighted rows indicating sources of 
data reported in this paper. Our efforts followed accepted best practices for assessing learning 
outcomes in international education (Deardorff, 2006), including through use of a concurrent 
mixed methods approach to collect and analyze both quantitative and qualitative data (Creswell 
and Plano Clark, 2011, Ch. 3). We also leveraged indirect sources of evidence, primarily surveys, 
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Data Source 

Before 

orientation 

During 

orientation 

At end of During 

orientation experience 

At or after 
re-entry meeting 

(a) Demographic survey 

y 




(b) Sojourn readiness assessment 

y 


/ 


(c) Self-reports of language proficiency 

y 



y 

(d) Diversity survey (MGUDS-S) 

y 



y 

(e) Global competency scenarios 

y 


/ 

y 

(f) Reflective activities and blogs* 


✓ 

y 

y 

(g) Orientation evaluation survey 



y 


(h) Program evaluation survey 




y 

(i) Interviews and focus groups 




y 

(j) Trip report* 




y 

(k) Research host/sponsor survey 




y 

(1) Final program survey 




y 

*Sample participant responses are available on GlobalHUB (http://globalhub.org/iree). 


Table 3. Data Collection Plan for Program Evaluation and Outcomes-Based Assessment. 


along with more direct measures, such as scenario-based tasks and reflective journal entries. Our 
findings begin by reporting on a number of more general aspects of global competence investi¬ 
gated using a variety of self-report survey instruments. These data sources include a new 15-item 
Sojourn Readiness Assessment questionnaire that we developed for the program, the 15-question 
Miville-Guzman Universality-Diversity Scale - Short form (MGUDS-S) (Fuertes et al., 2000), and 
self-reported levels of Chinese (Mandarin) language proficiency. We selected the MGUDS-S instru¬ 
ment due to its relative brevity, free availability for educational and research purposes, coverage 
across three domains, and demonstrated validity and reliability. Additionally, we were aware of 
prior uses of MGUDS-S in engineering education, potentially allowing us to compare results with 
other relevant student populations. 

Given this paper’s significant focus on global engineering competency, we also report data from 
orientation and program evaluation surveys, a scenario-based global competency question, and re¬ 
entry interviews. While the survey questions asked participants to self-report levels of competence 
in specific areas, the scenario question and interviews are potentially more reliable sources of data 
since they make it less likely that respondents can fake or exaggerate their responses, misunderstand 
question statements, and/or respond to questions quickly and thoughtlessly. During the interviews, 
we also specifically probed participants’ observations about how their Chinese colleagues defined 
and solved technical problems by using prompts such as: “Did your Chinese colleagues ever approach 
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or solve technical problems in ways that were unusual or different from what you expected? Can 
you give any specific examples?” 

When possible, paper forms were used for data collection. The Qualtrics survey platform was used 
for data collection in other situations (e.g., surveying the online orientation group, and administering 
the host/sponsor survey and final program survey). In all cases, appropriate data collection procedures 
were approved and followed under Purdue IRB protocol #1004009220. All quantitative data for this 
study was compiled and analyzed in Microsoft Excel. Qualitative interview data was organized and 
analyzed using the Dedoose application. The specific procedure involved a group of researchers using 
a small set of a priori codes to identify passages in the interview data relevant to global engineering 
competency. And while this paper is unique given its primary focus on global engineering competency, 
in prior publications we have reported select MGUDS-S, global scenario, and program evaluation results 
for the IREE 2010 China program (Jesiek et al., 2011; Jesiek & Woo, 2011; Jesiek et al., 2012). 


FINDINGS 


Our findings report learning outcomes from IREE from lesser to greater specificity, beginning 
with perceptions of readiness for a sojourn abroad, foreign language abilities, and other aspects 
of cross-cultural competence. We then turn to a more in-depth review of findings related to global 
engineering competency based on three main sources of data: a set of self-evaluation questions, 
an open-ended global scenario question, and post-experience interviews. 

Sojourn Readiness Assessment 

A 15-item Sojourn Readiness Assessment (SRA) instrument was developed to evaluate IREE 
awardees’ general sense of preparedness for a sojourn abroad. Sample survey items are presented in 
Table 4, and were ranked by respondents on a six-point Likert scale ranging from Strongly Disagree 


Dimension 

Sample Question 

Cognitive Preparedness 

I have adequate knowledge about the host country. 

Doubt 

I question if going abroad was a good decision.* 

Perception of Benefits 

The benefits of going abroad outweigh the challenges. 

Anxiety 

I am anxious about going abroad.* 

*These items are reverse 

scored. 


Table 4. Sample Sojourn Readiness Assessment (SRA) Questions. 
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Group 

n 

IREE Pre-Orientation 
Average Total Score 

IREE Post-Orientation 
Average Total Score 

Score 

Change 

Purdue 

17 

64.4 

68.7 

+4.4 1 

Shanghai 

18 

63.9 

69.1 

+5.1 2 

Cyber 

17 

67.4 

67.3 

-0.1 

All 

52 

65.2 

68.4 

+3.2 1 


^core change is statistically significant (p<0.01) based on a paired t-test. 
2 Score change is statistically significant (p<0.05) based on a paired t-test. 


Table 5. IREE Pre/Post-Orientation Sojourn Readiness Assessment (SRA) Results. 


(1) to Strongly Agree (6), giving the instrument a total scoring range of 15-90. The SRA was ad¬ 
ministered to participants at the beginning and end of their respective orientation program. As 
summarized in Table 5, the average total SRA score rose from 65.2 before orientation to 68.4 after, 
representing a statistically significant increase. Further analysis indicates no significant change in 
scores for the cyber/online orientation group, while the two face-to-face groups saw notable gains. 
We propose two explanations for these trends. First, many participants opted into the cyber/online 
group because they were already more prepared for a research sojourn, as indicated by their higher 
incoming SRA scores. Second, these participants were less likely to complete all of the required 
training, and what they did complete occurred with fewer opportunities to engage with and learn 
from their peers and the program staff. 

Factor analysis of our results also allowed identification of four specific dimensions potentially 
measured by this instrument, namely Cognitive Preparedness, Doubt, Perception of Benefits, and 
Anxiety, as shown in Table 4. Based on these findings, a new 20-item version of the survey is cur¬ 
rently being piloted. 

Foreign Language Skills 

Our assessment of foreign language ability (Mandarin Chinese) was based on a five-level self-report 
survey and scale originally developed by McNeil (2010). Table 6 presents pre/post-IREE data that was 
collected at the start of orientation and again at the re-entry meeting. In summary, 26 of 51 (or 51%) of 
respondents indicated an increase in language proficiency, with the average level increasing from 2.2 
to 2.8. Additionally, the number of participants who indicated they could “engage in basic conversa¬ 
tion” doubled, from 11 to 22. While many methods are available to more rigorously evaluate language 
capabilities (e.g., language proficiency tests and exams), our data nonetheless suggest notable gains 
in basic language skills among IREE participants. Our qualitative data (e.g., reflection journals and 
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Level of Language Proficiency (Mandarin Chinese) 

Pre-IREE 

Post-IREE 

(1) No proficiency 

16 

0 

(2) I know a few words a phases 

18 

22 

(3) I can engage in basic conversation 

11 

22 

(4) I could take engineering courses in this language 

1 

2 

(5) This is my native language 

5 

5 

Total 

51 

51 


Table 6. IREE Participant Self-Reports of Language Proficiency (n = 51). 


interviews) also revealed a handful of participants who were extremely diligent with their language 
training efforts during their sojourn, including those who actively sought out opportunities for further 
formal and informal language practice through hired tutors, conversations with peers, etc. 

Universal-Diverse Orientation (UDO) 

The Miville-Guzman Universality-Diversity Scale - Short Form (MGUDS-S) was used to evaluate 
aspects of cross-cultural competence among participants, including possible changes pre/post- 
IREE. This 15-item survey measures universal-diverse orientation (UDO), or “an attitude of awareness 
and acceptance of both similarities and differences that exist among people” (Miville et al., 1999). 
The instrument features three subscales that assess the extent to which respondents have diversity 
of contact with others (behavioral), relativistic appreciation of oneself and others (cognitive), and 
emotional comfort with differences (affective) (Fuertes et al., 2000). 

MGUDS-S was administered to all participants before orientation and again at the re-entry 
meeting. As reported in more detail elsewhere, data analysis reveals small but significant pre/post 
increases in average MGUDS-S total scores, rising from 73.6 to 76.4 on a 90-point scale. This result 
suggests that the IREE experience had a positive impact on participants’ UDO. Further analysis 
also revealed different impacts by sub-group (Jesiek et al., 2012). For instance, women tended to 
enter the program with higher UDO and had larger gains as compared to men, and the largest pre/ 
post-IREE gains occurred among men with prior experience living abroad, and women without 
such experience. This suggests a possible priming mechanism at work, with men needing multiple 
immersive travel experiences to enhance their cross-cultural competence. 

As indicated in Table 7 below, an increased sense of cross-cultural competence was also evident 
in IREE student responses to select re-entry survey questions. For example, 55 of 55 respondents 
agreed or strongly agreed with the statement “I will be able to work more effectively in diverse and 
multicultural environments.” Additionally, when research hosts were asked about the cross-cultural 
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skills of the IREE students who had worked in their labs, the respondents (n=34) gave an average 
rating of 4.32 on a 5-point scale ranging from Unsatisfactory (1) to Outstanding (5). 


Global Engineering Competency: Self-Evaluation 

Two primary strategies were used to investigate the global engineering competency of IREE partici¬ 
pants. First, a series of self-evaluation questions were given at the re-entry meeting as part of a final 
program evaluation survey. The questions were in part based on prior work by Downey et al. (2006), 
which investigated learning outcomes among students in the Engineering Cultures undergraduate 
elective course at Virginia Tech. As summarized in Table 7, all but one question had average scores in 
the 3.3 to 3.5 range on a 4-point scale ranging from Strongly Disagree (1) to Strongly Agree (4). For 
the three questions that can be directly compared to results from students at Virginia Tech, we find 
identical average scores for statements (c) and (e). We suspect that notably higher scores among 
IREE students for statement (f) reflect enhanced self-efficacy resulting from an immersive global 
work experience, especially as compared to the non-travel experience at Virginia Tech. It is further 
worth noting that the statement “I gained significant knowledge about engineers and engineering in 
China” received the lowest average marks (3.1). We posit that this is due to the fact that participants’ 
exposure to engineering in China was limited to interactions with small numbers of engineering stu¬ 
dents and faculty members, usually in one lab and focused on a specific research area. 


(maximum n=55) 

Strongly 

Disagree 

(1) 

Disagree 

(2) 

Agree 

(3) 

Strongly 

Agree 

(4) 

IREE 

Avg. 

Score 

VT EC 
Avg. 
Score* 

a. I gained significant knowledge about engineers and 
engineering in China 

0 

9 

33 

12 

3.1 

N/A 

b. I am better prepared to meet and work with 
engineers from China 

0 

0 

26 

29 

3.5 

N/A 

c. I am better prepared to meet and work with 
engineers from different countries 

0 

1 

29 

25 

3.4 

3.4 

d. I will be able to work more effectively in the global 
engineering profession 

0 

0 

25 

30 

3.5 

N/A 

e. I now have a better understanding of how my 
perspective as an engineer is different from those of 
engineers from other countries 

0 

2 

27 

26 

3.4 

3.4 

f. I will now be better at working with people who 
define problems differently than I do 

0 

1 

36 

18 

3.3 

3.1 

g. I will be able to work more effectively in diverse 
and multicultural environments 

0 

0 

26 

29 

3.5 

N/A 

*These results originally reported by Downey et al. (2006). 


Table 7. IREE Participant Self-Evaluation of Global Engineering Competency. 
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Global Engineering Competency: Scenario-Based Assessment 

Utilizing a scenario-based assessment strategy (Jesiek and Woo, 2011) and building on previous 
work by Downey et al. (2006), the open-ended global scenario shown in Figure 1 was also used to 
assess the global engineering competency of participants before and after their IREE experience. 
Created by members of the lead author’s research group, the scenario is designed to measure 
cultural sensitivity in the context of a realistic global engineering work situation, namely by juxta¬ 
posing a serious technical issue (i.e., a quality control problem) against disagreement with one’s 
Chinese supervisor. The four-level scoring rubric shown in Figure 1 was created based on pilot sce¬ 
nario data collected from talk-aloud interviews with four subject-matter experts (engineers with 
extensive experience working in industry or academic settings in China) and written responses 
from 44 “novices” (first-year engineering students). The first three levels of the rubric reflect dif¬ 
ferent levels of cultural sensitivity, while the fourth level adds a cognitive or knowledge dimension, 
namely understanding how to “save face” with one’s superior and colleagues in a Chinese work 
setting. Two researchers used this rubric to analyze all pre/post-results for the IREE students, and 
all coding discrepancies were resolved on a case-by-case basis through discussion until consensus 
was reached. 

As indicated in Figure 1, the average pre/post scores on this task rose from 1.62 to 1.96, rep¬ 
resenting a statistically significant increase (Jesiek and Woo, 2011). And while this task tends to 
elicit ambiguous responses that are difficult to score, it is notable that the number of respondents 


Scenario: As an employee in a large multinational corporation, you are temporarily assigned to your company’s 
branch operations in Shanghai, China. Your work team consists of three Chinese engineers, all at about the same rank 
as you. Your team reports to an engineering manager, who is also Chinese. In a recent team meeting, your manager 
presented a solution to a difficult quality control problem. However, you feel the proposed solution will likely fail. 
How would you handle this situation, and why would you handle it this way? 

Response Level / Description / Count 


n-45 

valid 

data 

pairs 

Clear Evidence 
of Cultural 
Insensitivity or 
Inappropriateness 

Cultural 

Sensitivity Unclear 
(Response is Neutral 
or Ambiguous) 

Clear Evidence of 
Predisposition to 
Cultural Sensitivity 
(Not Context Specific) 

Clear Evidence of 
Cultural Sensitivity, 
Including Knowledge 
of Specific Context 

Avg. 

Score 

Pre-IREE 

0 

25 

12 

8 

1.62 

Post-IREE 

0 

23 

1 

21 

1.96 1 


’Difference between pre/ post-IREE average scores statistically significant (p<0.05) based on a Wilcoxon signed-rank 
test. 


Figure 1. Global Engineering Competency Scenario with Pre/Post Results. 
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Student Response, Pre-IREE (May 2010) 

Student Response, post-IREE (September 2010) 

““I would first discuss my opinion with my other these [sic] Chinese 
teammates (assuming we can communicate). If they agreed with my belief 
about the failure, then I would suggest we approach the manager as a team 
and confront him. If they disagreed, or suggested that it was a bad idea to 
confront the manager, I would still confront him especially if I felt SURE 
that the product would fail. If the team command me otherwise, I would 
obviously not go through with it (as long as their reasoning was based on 
sound engineering/science).” 

“Even though Chinese people (particularly those 
of high rank) are constantly concerned about 
‘losing face’, I think issues of quality control 
need to be addressed upfront and to the point. I 
would definitely approach the manager one-on-one 
though, so that the issue is just between he and I.” 

Score: 1 (Cultural Sensitivity Unclear) 

Score: 3 (Evidence of Cultural Sensitivity, Context 
Knowledge) 


Figure 2. Sample Pre/Post Responses to Global Engineering Competency Scenario. 


demonstrating context-specific cultural sensitivity (Level 3) increased from 8 to 21. Additionally, 
four of the IREE students shifted from level 1 to level 3 responses. To further illustrate this type of 
shift, Figure 2 presents pre/post responses for a single IREE participant. Before IREE this student 
responded to the scenario by placing primary emphasis on making an evidence-based technical 
decision, yet after IREE this tendency is counterbalanced with a desire to deal with the situation in 
a more culturally appropriate manner. We propose that such changes in one individual’s responses 
provide clear evidence of enhanced global engineering competency. 

Global Engineering Competency: Interview Findings 

Our re-entry interview data provides additional evidence of global competence among IREE par¬ 
ticipants. For example, some interviewees demonstrated both cultural sensitivity and context knowl¬ 
edge similar to what was expected for high levels of performance on the global scenario discussed 
above. When asked about adapting to the Chinese work environment, for example, one Ph.D. student 
described how he dealt with situations involving disagreements or competing approaches: 

Maybe if I’m in the U.S. and somebody in the research group brings up something, then 
maybe I see a problem, or I see, like maybe we should do it this other way, it would be more 
of a direct thing, say, “Oh but, you know, we could also do this and that.” But when I was 
in China I think I usually just held my tongue a lot more and would just, you know, try to 
observe and listen first and then, I don’t know, I guess depending on how the situation felt, I 
would approach it in a much more delicate way to, how to bring up my suggestion basically, 
you know. And, I guess, try to do it without making anybody lose face [...] or say that their 
plan is not good but say, “Well, your plan is also good but maybe, you know, maybe we 
could do it also, do this other part which is in addition to your plan.” 
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In addition to being aware of the importance of saving face in a typical Chinese work environment, 
this individual demonstrates effective cross-cultural communication strategies by being less direct 
and confrontational than what is typical in many U.S. settings. It is further worth noting that this 
same individual had participated in a study abroad program in China prior to IREE. As noted below, 
this hints at the importance of developing sensitivity to the conditions of non-native work contexts 
through multiple, lengthy immersions. 

The interviews also revealed that a number of IREE participants were able to describe how their 
colleagues in China approached technical problems differently, which we argue is another key marker 
of global engineering competency. Nonetheless, there was considerable variation in how they described 
these differences, including how they framed the relative merits of different approaches. For instance, 
one Ph.D. student criticized his Chinese colleagues for a tendency to both undertake "spin-off” studies 
and not interrogate fundamental principles and assumptions behind their research: 

People really wanted to look in the literature and see what the authoritative consensus was, 
and then they wanted to try and do a spin off of that. Their problem, there is some creativity 
in that obviously, but it’s really just applying what happened out there. You wouldn’t... almost 
ever [see] them sit down and brainstorm and say, “First principles, what’s happening here? 

What could be happening? What are the assumptions we’re making?” You know, and usually 
that’s a big source of your errors. Your assumptions were wrong or you're missing some 
underlying principle. Whereas they’ll take that all for set in stone because they followed it by 
the book in making [it] up to that point or they followed it by some outline in the literature 
or they’re looking for the out... and the answer to be that cliched phrase that you’re looking 
outside the box. You need to look at the bigger picture, need to look at the fundamentals. 

To account for these differences in style, this individual went on to discuss potentially relevant 
factors such as the kinds of experiences and interactions that are typical in U.S. universities: 

In the United States I think that’s something we’re trained to do maybe through university 
courses or just because of our individualistic nature. That we’re prone to opinions, we’re 
encouraged to discuss them. [...] You know, when you give presentations it’s very much 
like you give a talk but you discuss it. People quiz you. You really have to defend what your 
thoughts are. 

As these passages reveal, this student was clearly able to identify how Chinese researchers ap¬ 
proached problems differently and then posit possible underlying reasons for those differences. 
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However, such comments tend to frame U.S. research practices as superior to those observed in 
China. This tendency toward a kind of “engineering ethnocentrism” may limit the ability of this 
student to work in the midst of different perspectives and blind them to the potential advantages 
associated with alternate approaches to engineering research. 

Similar tendencies were observed among a number of other IREE interviewees, albeit to varying 
degrees. As one Master’s level graduate student explained: 

I don’t know if this is for everybody, but for our institution, our professor had so many 
people at his grasp, he just could be like I want you to try and solve this. So he would just 
send them into it without evaluating the situation, he would just say let’s do this. And in 
America, I think a lot of people would be like, alright, let’s take a step back, let’s look at the 
problems, let’s devise our best way of, try and figure out the most efficient way to try and 
solve these problems. And if we run into problems, let’s fix it at that time. Well, it seems like 
China was more like, I think trial and error is the best way to describe it. 

While this interviewee implies that the U.S. approach might have its advantages, he is cautious about 
overgeneralizing, including by declaring that this insight was based on observations made in just 
one local context. At other points in the interview, this participant also offered a further explanation 
for the observed style of research by discussing resource constraints, including how one’s approach 
might change depending on the size of a research team. In addition, this student explicitly moder¬ 
ated their remarks by adding: “I’m not going to say this is the correct way to do things, because 
you’re getting the same things done, just in a different way.” In contrast to the previous respondent, 
this student seems to have a somewhat higher level of understanding and respect for alternate ap¬ 
proaches to organizing and carrying out technical research. 

Nonetheless, only a few interviews revealed this kind of ability to understand and tolerate differ¬ 
ent technical perspectives. As one final example, a Ph.D. student succinctly described how Chinese 
researchers tended to use very different approaches to transportation research: 

I’m a lot more hands on. I want to go collect data and let’s do that. And for the most part, I 
mean, I haven’t I guessed worked on many of their projects with them. But, just seeing what 
they do is a lot more theoretical maybe. They want to look at like, we might be thinking of 
the same things. I’m interested in what factors influence your transportation but I’m going 
to go out and collect data and then try and build a model on that and then they want to 
analyze the more theoretical aspects. It’s maybe a little less hands on approach. For me, 
that’s kind of a big difference. 
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As this quote reveals, this student is explicitly cautious about overgeneralizing. And in the longer 
interview passage on this topic, there is no evidence of judgment about the superiority of more 
empirical versus theoretical approaches to transportation system modeling. 

The four students discussed here demonstrate an ability to identify and describe how their Chinese 
colleagues define and solve problems differently, albeit with different levels of sensitivity and toler¬ 
ance toward these variations. Yet who are these students, and how and why have they developed 
such abilities? To begin, all four interviewees profiled here are graduate students. In contrast to most 
undergraduates, these individuals have spent more time working in academic research settings in 
the US, making it easier for them to compare and contrast their domestic and global experiences. 
From a developmental perspective, they also tend to be more mature, and have likely been given 
more significant and meaningful roles in research projects, both at home and abroad. Finally, it is 
essential to note that all but the Master’s level student entered the IREE program after having already 
completed at least one extended (i.e., multiple months) research and/or study abroad experience in 
China. This finding suggests that the most marked improvements in global engineering competency 
may require multiple, lengthy exposures to work practices in diverse global settings. In addition to 
enabling development of cross-cultural competence in general, such experiences can also promote 
greater sensitization to local, national, and regional variations in engineering culture. 


CONCLUSION 


Responding to intensified globalization trends, many professional schools are internationalizing 
their courses and curricula to prepare graduates for careers that will likely involve working across 
countries and cultures. In professional fields such as engineering, medicine, business, and law, op¬ 
portunities for research, work, and/or service learning abroad are becoming more common ways 
to give students authentic professional experiences in global context. Yet these new programs and 
initiatives often require significant investments of time, money, and energy, and it is often unclear 
whether their actual impacts live up to the goals and objectives promoted by their champions. Of 
particular note is the challenge of systematically demonstrating how certain kinds of experiences 
and program formats support attainment of discipline-specific competencies needed by global 
professionals. 

Our results from the IREE 2010 China program contribute to a growing body of literature discuss¬ 
ing specific strategies for developing and assessing effective global learning experiences. And while 
our paper is mainly concerned with engineering, our work is intended to provide inspiration and 
insights that are relevant to other professional fields. More specifically, we present the Engineering 
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Cultures China curriculum as one essential part of a comprehensive orientation strategy that at¬ 
tempts to align the major program objectives and learning outcomes with appropriate instructional 
content and learning scaffolds. The content and associated activities were intentionally designed to 
support attainment of global engineering competency, including the ability to work with colleagues 
who define and solve problems differently due to individual and geographic variability in profes¬ 
sional identities, educational backgrounds, organizational contexts, and sociocultural norms. We 
also propose some other ways to help scaffold and support experiential learning abroad, including 
critical incident reflection prompts, and re-entry activities like interviews, focus groups, and trip 
reports. Many of these approaches can be adapted for use in other courses and programs, both 
within and beyond engineering. 

The IREE program has also inspired ongoing efforts to enhance the Engineering Cultures China 
curriculum. For example, the scenario prompt presented in Figure 1 inspired the lead author to write 
a longer script that more completely illustrates how this type of work situation might play out in 
a real-world setting. The script was acted out by a group of graduate students, and the resulting 
video clips were edited to create Global Engineering Competency Vignette #1, as shown in Figure 3 
Video link: https://www.voutube.com/watch?v=K5mA7yb14Gq&index=2&list=PL84VcUp5xaBsRutUS- 
8KR2lwFxh8Kfl9G . This brief video (less than three minutes in length) is intended for use by instruc¬ 
tors who wish to seed and facilitate case-based conversations about typical situations encountered 
in global engineering work, particularly in China and other countries in East Asia. The video and an 
instructor’s guide are freely available on YouTube (Jesiek, 2013). 

Finally, this paper represents an important step toward developing more robust and scalable assessment 
strategies and frameworks for evaluating key learning outcomes that are uniquely or especially relevant 
for practicing engineering in cross-national/cultural contexts. More specifically, we describe a mixed- 
methods approach leveraging multiple direct and indirect measures. In addition to summarizing results 
related to sojourn readiness, foreign language proficiency, and universal-diverse orientation, we report 
data from self-assessment questions, an open-ended scenario prompt, and re-entry interviews as evidence 
of enhanced global engineering competency and related outcomes among IREE participants. 

In general, our findings strongly resonate with results from the large-scale, multi-year Georgetown 
Consortium Project, which found that learning is maximized in well-structured programs with pro¬ 
active learning interventions, and with the greatest gains in intercultural learning occurring among 
students who spend roughly a semester abroad (Vande Berg, Connor-Linton, & Paige, 2009). The 
present study also tentatively suggests that the largest gains in global engineering competency oc¬ 
cur among students who have had multiple, in-depth exposures to different cultures of engineering. 
Such findings are important for program faculty and staff to consider as they design, deliver, and 
evaluate global learning experiences. However, additional research is needed to identify and better 
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Figure 3. Screen Capture from Global Engineering Competency Vignette #1 (Jesiek, 2013). 

Video link: https://www.voutube.com/watch?v=K5mA7yb14Gq&index=2&list=PL84VcUp5xaESsRutUS- 
8KR2lwFxh8Kfl9G 


understand the specific kinds of critical experiences that maximize learning gains and development 
of key competencies. 

The limited scalability of open-ended scenarios and participant interviews also suggests the 
need for more work on assessment tools and strategies. Given such challenges, the lead author’s 
research team has undertaken a new research project that involves refining our definition of global 
engineering competency. This effort will in turn support development of a valid and reliable Situ¬ 
ational Judgment Test (SJT) that can be used to assess 3-4 major dimensions of global engineering 
competency in six national contexts. We expect that creating such tools will help scaffold ongoing 
efforts to understand and enhance how students and professionals are prepared for the global 
realities of engineering practice. 
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